1. Field of the Invention
An embodiment of present invention relates to a depth-based image rendering method, medium, and system, and more particularly, to an image-based rendering method, medium, and system that blends splats disposed within a certain offset value from a three-dimensional surface of depth-based image data with one path.
2. Description of the Related Art
Image-based rendering generates a resultant image from a random viewpoint based on three-dimensional space data of an original certain standard viewpoint (or camera). In such image-based rendering, since a real image can be input, a derivable resultant image may be generated that looks like the same real image but from a different viewpoint.
One type of image rendering is a three-dimensional depth image-based rendering method, where a resultant image is rendered by using original image information and measured spatial depth values for points and/or pixels within the original image.
In such image-based rendering, a warping process is performed to transition the original pixels, if necessary, of the original image to a new position or to have different characteristics in the resultant image. Here, since the size or shape of an object of the original image may change during warping, resulting in the generation of displayed holes or aliasing between warped objects, methods of improving image quality by optimizing the size and shape of resultant pixels of the resultant image are currently being developed.
For example, one method includes blending, by applying Gaussian filtered disk shaped splats of original image information. In this case, two conventional representative techniques include using a z-offset inspection with two paths and using a McMillan algorithm with one path.
FIG. 1 illustrates an example of a depth image-based rendering method to blend color in a depth direction according to such a conventional methodology where blending is performed by applying splats for a resultant image based on the z-offset inspection with two paths.
First, a depth map is generated by comparing depth information values with respect to splats included in all generated surfaces/layers to be used in forming the image. This depth map may initially include depth information for different objects of pixels seen in an original viewing of a scene or image area, e.g., pixels of objects that are part of the background may have greater depth values from a defined zero depth and pixels of objects that are closer will have lower depth value, and possibly a negative depth value.
A splat is a modifiable component, representative of warpable data for an original pixel, that can be applied to each surface forming the resultant image and can be used as a technique to transpose or warp a pixel from a standard pixel to an oblong or warped shape, or even to spread the pixel out over several resultant pixels as the resultant image is generated for a new vantage point. These splats can be generated from different surfaces and can be blended with other image data. For example, a final resultant pixel used for display of an image from a new perspective, e.g., a new view angle or direction, may be defined by several splats on different surfaces at different depths at that pixel location, with each splat potentially originating from different original pixels that have been warped according to the new perspective. When the splat information from the different surfaces is combined, the color, luminance, and/or opaqueness of the final resultant pixel can derived. The surfaces are a dimensional or layering mechanism that helps in defining the ‘depth’ of the image to more easily generate the final resultant image from the new perspective. Splat information at different depths may be defined on different surfaces. A more clear understanding of this concept will be found after the following discussion of these conventional methods.
Accordingly, similar to that noted above, after the warping of the splats has been done a resultant image can be put together by collecting the different surfaces, e.g., based on a depth information value of a splat computed from a difference between a distance from a central pixel on one surface to a rendering viewpoint and a distance from the rendering viewpoint to different respective splat information on different respective surfaces of the image, i.e., it's new perceived depth.
As another example, respective depth information values for putting together the resultant image, with respect to each splat included in all respective surfaces forming the image, may be generated by using the difference between a distance value from a rendering viewpoint 101 to each respective pixel included in the image and a distance value from the rendering viewpoint 101 to a respective surface 102 for the image.
In this case, using FIG. 1 as an example, the respective depth information values for a splat included in each surface may be generated in a sequenced order from an outer surface 103, furthest from the rendering viewpoint 101, to the most inner surface 102, closest to the rendering viewpoint 101, namely, in the direction of the first path 104.
Secondly, in this example, the depth information values with respect to each splat are stored in a buffer, thereby generating the depth map for the resultant image, also permitting searching for a standard splat closest to the rendering viewpoint 101 by referring to the depth map.
Third, if a calculated depth for a splat is not greater than an offset value then the corresponding information for that splat is applied to that existing surface, e.g., through a blending by referring to the depth map.
The offset value indicates a certain depth value for a surface of for the resultant image, e.g., with the depth value being based upon a depth relative to the surface closest to a viewpoint. For example, the offset value of FIG. 1 may represent a certain depth value 106 from the surface 102 of for the resultant image.
When putting the resultant image together, the depth information values of the splats disposed along the second path 105 are then read from the buffer, and it may be determined whether the depth information values represent a particular depth away from surface 102, for example, by less or equal to the offset value. For depth information values that are not greater than the offset value, color information values of the splats can be read from a buffer and information for splats can be blended and stored in a buffer.
This described z-offset inspection blending using the two paths has an advantage of improving image quality due to the blending effects. However, there is a disadvantage in that the blending speed is very slow because two paths are required, i.e., first depth information values are calculated from the illustrated 103 surface to an innermost surface along path 104 and the blending is the performed along path 105. In particular, blending speeds become even slower as image resolutions increase.
In order to overcome such disadvantages, an alternate method using the McMillan algorithm with one path has also been implemented.
FIG. 2 illustrates the other example of a conventional depth image-based rendering method. Referring to FIG. 2A, in the McMillan algorithm using one path, blending is performed by apply blending to all of the image data.
According to the McMillan algorithm method, when generating a resulting image, color information values with respect to splats included in all surfaces forming an image are read from a buffer, in the order from the surface 203 furthest from a viewpoint 201 to a surface 202 closest to the viewpoint 201, information for the splats are sequentially blended, and the color information values of the blended splats are stored in the buffer.
Since the described McMillan algorithm uses one path, the blending speed may be improved. However, since information on the closest surface is acquired last, an offset designating a distance from the surface closest to the viewpoint cannot be determined before blending splat, thereby causing image degradation or mixing of image information with background or hidden parts of the image.
Namely, since a splat is blended without comparing depth information values of the splats during the blending, as shown in FIG. 2B, blending is performed between splats that are separated from each other more than a certain distance, such as the first splat 205 and second splat 206. When the second splat 206 corresponds to a background or covers parts of image data, differently from the first splat 205, the second splat 206 also is blended, thereby generating the described image degradation.
FIG. 2C thus, shows an example of a resultant image that has been generated through blending with the McMillan algorithm. Here, image degradation in the resultant image occurs due to blending based only on splats separated from each other more than a certain distance.
Accordingly, there may be a desire for a depth image-based rendering method, medium, and system with improved blending speed, improved by using one path and a blending range to improve the quality of a resultant image.